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(54) Method and system for geometry measurement 

(57) There is disclosed a method for determining 
the position and/or orientation of a number of objects Rg.3 
relative to each other. The method includes the steps of 
providing at least two cameras (1,8) and a probing tool 
(15). together with a network of help reference points 
constituted by active light sources (16), reflective points 
(18) or points of measurement determined by use of the 
probing tool. The help reference points can also be 
embodied as existing points or geometrical objects (14) 
whose position is known. 

The method also includes a determination of the 
positions of selected elements of the points in the net- 
work relative to each other by using at least one of the 
cameras (1 ,8) to determine the location of a probing tool 
(15)in contact with the point. The camera positions are 
also determined. From these data, the positions of the 
selected objects relative to each other may also be 
determined. 
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Description 

The present invention relates to a method, as welt 
as a system for determination of relative position and/or 
orientation of a number of points or geometrical objects s 
as planes, lines, holes, cylinders or spheres, or com- 
bined objects as robot arms or other industrial produc- 
tion equipment, based on the use of one or multiple 
cameras based on electro optical sensors in combina- 
tion to fixed mounted fight sources, projected light spots. 10 
illuminated reflectors or probing tools having light 
sources or reflectors attached to it. 

The present patent application describes a further 
development of the inventions described by the inven- 
tors in Norwegian patents 165 046. 164 946 and 169 is 
799. as well as patent application 913994. 

Norwegian patent 165 046 describes a system 
based on two high resolution electro optical cameras 
. which are calibrated for angle measurements, methods 
for such calibration, a system for geometry measure- 20 
merits based on the cameras, as well as application of 
this system. 

Norwegian patent 164 946 describes systems for 
measurements of points on a surface, by projecting a 
pattern of light spots onto the surface, and by determin- 25 
ing the coordinates for these light spots by the use of 
electro optical sensors. 

Norwegian patent 169 799 describes tools for 
marking points on a surface, to be measured by the use 
of electro optical cameras. 30 

Norwegian patent application 913994 describes a 
system for measurement of spatial coordinates by the 
use of a single electro optical camera as well as tools 
having light source or reflecting points attached to it 

The system that is described in Norwegian patent 35 
1 65 046 has limitations related to the fact that each indi- 
vidual point is registered simultaneously by two cam- 
eras. That leads to requirements for simultaneous free 
visibility in directions to both cameras. Furthermore the 
system is based on determination of the position of the 40 
cameras by a separate procedure before coordinate 
measurements can be started. The system accuracy is 
basically determined by the resolution of the sensor. 
This means that the accuracy is limited in large vol- 
umes. 45 

The system described in Norwegian patent applica- 
tion 913994 functions essentially as a theodolite, in that 
spatial directions are determined with high accuracy. In 
addition the distance to the measurement tool-is deter- 
mined with moderate accuracy. As a stand alone sys- so 
tern it gives limited accuracy in distance determination, 
which means that the number of possible applications is 
limited. 

The use of a probing tool having light sources is 
known from WO-A-91/16598. 55 

The present patent application describes a further 
development of method and means for coordinate 
measurement that gives full flexibility with respect to: 



the number of cameras, 

whether the cameras are calibrated for direction 
measurements or not, 

the methods used for marking the measurement 
points, 

where, and in how many positions the cameras are 
located during the measurement process, 
- which methods that are used to relate the measure- 
ments to a selected coordinate system. 

By the use of this further development full flexibility 
to optimize the system and the method with respect to 
the measurement task in question, and to questions like 
accuracy requirements and exterior limitations like visu- 
ally limitations is achieved. 

Regarding applications, the system is competing to 
existing technology like theodolites and conventional 
photogrammetry, as well as mechanical coordinate 
measurement machines. 

Coordinate measuring machines have strongly lim- 
ited functionality, as they are not portable and have a 
limited measurement volume. Large machines are very 
expensive, have low accuracy, and require dedicated 
rooms due to the need for stability and temperature con- 
trol. 

Theodolites have significant limitations related to 
operator dependencies (in conventional instruments an 
operator is aiming towards the measurement point 
through the monocular of the theodolite), time consum- 
ing measurements, and to strong requirement for free 
visibility between the measurement point and the instru- 
ment. In addition theodolites must be levelled accu- 
rately. 

Conventional photogrammetry is limited to the use 
of film based cameras. The measurement points are 
marked by attaching reflecting measurement elements 
called targets, to the object. It is not possible to measure 
geometrical objects like holes, planes, spheres, lines, 
etc. without marking these by the use of targets. 

Calculation of spatial coordinates based on theodo- 
lite measurements or conventional photogrammetry is 
done by iterative methods. Both measurement methods 
are two dimensional, i.e. they give the direction towards 
the measurement points only. Estimated spatial coordi- 
nates for a number of the measurement points are 
needed as initial values for the iterative calculations. 
The system described in Norwegian patent application 
913994 gives spatial coordinates, which gives suffi- 
ciently accurate initial values for the similar calculations 
related to the present method and system. 

Figure 1 illustrates camera and imaging of a light 
spot on the sensor of the camera. 

Figure 2 illustrates the principle of coordinate deter- 
mination from the imaging of a common 
point in two or more cameras. 
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Figure 3 illustrates the configuration of a complete 
measurement system. 

Norwegian patent no. 165 046 describes a fully 
automatic and accurately calibrated angle sensor based s 
on a CCD camera. Such a camera 1 . as shown in figure 

1 . comprises essentially a camera house 2, a lens unit 
3, and a two dimensional sensor matrix 4. The lens unit 
is an objective with standard, spherical optics. The 
angle sensor is developed to measure the direction w 
towards points being active light sources or points illu- 
minated by active light sources. This gives a secure 
identification of the measurement points, and hence 
enables a fully automatic use. as well as gives a very 
high signal to noise ratio. High accuracy is furthermore is 
ensured by the use of accurate procedure for calibration 

of the angle sensor. This is described in Norwegian pat- 
ent 165046. 

The measurement principle is illustrated in figure 2. 
The coordinates for a number of points 5 shall be deter- 20 
mined relative to a spatial coordinate system X, Y. 2. 
This is done by locating two or more cameras 4 in arbi- 
trary and initially unknown positions and orientations 
relative to the same coordinate system. The spatial 
direction to a point 5 is determined by registering its pro- 2s 
jection 6 through the projection center 7 of the lens unit 
as shown in figure 1 B. The projection is registered as 
the image coordinates x. y of the point image relative to 
a camera fixed coordinate system as illustrated in figure 

2. The calculation of the unknown spatial coordinates is 30 
based on setting up the equations for the projections for 

a number of cameras. This calculation may also include 
determination of the position and orientation of the cam- 
eras, as well as including parameters for correction of 
possible lens distortions. This means that even non-cal- 35 
ibrated cameras may be applied. Each additional point 
that is introduced in the calculation gives three addi- 
tional unknown coordinates X, Y, 2 to be determined, at 
the same time as 2 x n new observations are included in 
the calculation (n is the number of camera locations). <o 
The calculation requires the magnitude of at least one 
distance between two points to be known to give correct 
scale information to the system of equations. 

The method of calculation is based on minimizing 
errors (least squares method), such that redundant 45 
information is used. The necessary mathematical foun- 
dation is found in H.M. Kamara (Ed.): Non-topographic 
photogrammetry. Second Edition. 1987. page 37-55. 

Generally, if the number of unknown parameters is 
increased, a similarly increased number of observations so 
as additional points or additional cameras/camera loca- 
tions is required. 

Figure 3 illustrates a system for spatial geometry 
measurements based on one or multiple cameras 1 . 8. 
a system control unit 9 consisting of data processor 10. ss 
camera control unit 11 and light source control unit 12. 
The system can be attached to different light sources for 
marking points: 
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Reference bar 14 comprising a number of light 
emitting diodes at known mutual separation dis- 
tances. 

Light pen 15. which is a probing tool described in 
Swedish patent no. 456 454. posstoly having 
exchangeable tools as described in Norwegian pat- 
ent no. 169 799. 

Light emitting diodes and/or lasers 16. Light emit- 
ting diodes are attached permanently in the meas- 
urement field to be imaged from multiple camera 
locations. Lasers are used to project fixed light 
spots onto the measurement object. Light emitting 
diodes / lasers are connected to the system via a 
connector urat 17. 

Reflecting points 18 which are illuminated by a 
lamp 19. 

Driving and control of the light sources is done by 
the control unit 12 to ensure optimum signal to noise 
ratio, and to synchronize to the cameras via the camera 
control unit 1 1 . 

The systems data processor 10 is doing the analy- 
sis of measurement data from the cameras. The data 
processing mainly consist of: 

control of imaging time and exposure time to opti- 
mize the signal to noise ratio. 

identification of the individual light sources, i.e. 
which point in the image that corresponds to which 
light source. 

calculation of the spatial direction for each individ- 
ual light source from the image information. 

- calculation of spatial coordinates for the probing 
tool (light pen). 

The user is communicating with the system via an 
operator terminal 13. 

In addition to the observations from the cameras, 
the method of calculation requires one or more known 
distances to give the correct scale to the calculated spa- 
tial coordinates. This can be achieved by the use of the 
specially designed reference bar 14, or by measuring 
points in a known separation distance. The length of the 
reference bar or the known distances may be typed in 
by the operator at the terminal 13. 

The present invention proposes to apply one or 
more cameras in combination with one or more types of 
light sources or illuminated reflecting points to achieve 
high flexibility to solve various measurement problems 
with respect to 

- the dimension of the measurement volume. 
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the accessibility to the measurement volume, 

definition of coordinate systems, 

accuracy requirements, 

available time for measurements, 

frequency of repeated controls, 

combination of different types of cameras and 
instruments for direction measurements. 

The characteristic features of the invention are 
described in the accompanying patent claims, as well 
as in the following description of non-limiting examples 
of use of the invention, with reference to the enclosed 
drawings. 

Figure 4 illustrates the further developed 

principle for the use of the probing 
tool light pen". 

Figure 5 illustrates how to establish and 

measure a network of help refer- 
ence points. 

Figure 6 illustrates measurement of a 

number of geometrical objects 
based on a number of help refer- 
ence points. 

Figure 7 illustrates an application based on 

a transportable reference structure 
/pallet. 

Figure 8 illustrates a permanent measure- 

ment station based on a reference 
structure. 

Figures 9 and 10 show examples of use of the 
method of the invention. 

The computation method that is described above 
and that is related to figure 2. is based on the possibility 
of seeing the same point from a number of different 
camera positions. Generally light sources, generated by 
direct emission from a light emitting diode or indirect as 
a reflection from a reflecting material or a projected 
laser spot, can be seen from "one side* only. Optimum 
accuracy in the computation method is achieved if the 
directions of observations are uniformly distributed. This 
is achieved by the light pen. 

The light pen 1 5 that by itself is known from Norwe- 
gian patent application 913994. is in this context used 
differently as only the direction to the touch point of the 
light pen is to be registered and to be used for coordi- 
nate determination. The principle for use of the light pen 



is shown in figure 4. The light pen is registered on the 
sensor as a number of image spots 6 corresponding to 
the light emitting diodes 5. The direction to the light pen 
should be related to a touch point 20. This corresponds 

5 to an image point 21 on the sensor. As the sensor is reg- 
istering the image of light sources only, this will be a vir- 
tual image spot. The image coordinates of the image 
point 21 will be calculated from the registered image 
coordinates for each light source, as well the Knowledge 

io of the mutual position of the light sources 5 and the 
touch point 20 in a probe fixed, local coordinate system. 
The calculation methods are further described in Nor- 
wegian patent application 913994. 

If the touch point of the light pen is kept in a fixed 

15 position, the light pen itself can be rotated to be aimed 
towards the different camera positions in question. 
Hence, this gives the necessary geometry for the com- 
putation method. The fact that the light pen has a 
number of light emitting diodes makes the accuracy in 

20 determining the projection of the touch point better than 
for a single light emitting diode. 

Figure 5 illustrates the use of the system to deter- 
mine the geometry of an object 22. The positions of a 
number of points or geometrical objects (holes, planes. 

zs cylinders etc.). in the figure marked with capital letters A 
- C, are to be determined with high accuracy relative to 
the resolution of the cameras, the size of the measure- 
ment volume, its accessibility etc. The method consist 
of distributing a number of help reference points (in the 

30 figure marked with lower case letters a - u) all over the 
measurement volume. These can be light emitting 
diodes, projected laser spots, reflecting points or 
marked points that can be touched with the light pen. By 
registering the projection of these points on the camera 

35 sensors for a number of different camera locations, this 
will give data for calculation of the mutual position of all 
points, as described above with reference to figure 2. 
The computation is flexible with respect to: 

40 - the number of cameras or camera locations, 

the number of points observed in each camera 
location (a minimum number is required, depending 
on the number of camera parameters to be detfer- 
45 mined), * 

the number of observations tor each individual 
point. 

so In general the accuracy is improved if the number of 
observations (points) in each camera location is 
increased, and if the number of camera locations is 
increased. A single camera or multiple cameras can be 
used. If the cameras are not calibrated with respect to 

55 the imaging properties of the lens, it is advantageous to 
have as few different cameras as possible. 

If, on the other hand, it is necessary to do the meas- 
urements last the data acquisition can be made effi- 
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cient by using multiple, calibrated cameras in fixed 
locations, and by using as few locations and points as 
possible. 

The relation to an object fixed coordinate system 
requires some points that are defining the coordinate 
system by their coordinate values being known along 
one or more of the coordinate axes, or by doing a best 
fit of defined values to the calculated positions. 

Figure 6 illustrates how to start from a number of 
points in known positions relative to an object fixed 
coordinate system and then to measure additional 
points within or outside the object by the use of two or 
more cameras. If a network of points is available, as 
shown by letters a - u, it is possible by the use of only 
two cameras 1 .8 to locate these such that they are see- 
ing not more than the area of interest At first the posi- 
tion and orientation of the cameras relative to the object 
fixed coordinate system are determined by registering 
the projection of the known points a. b. c. The coordi- 
nates of these points relative to the coordinate system 
X. Y are known from a measurement as described 
above with reference to figure 5. This method gives a 
quick measurement, having a high accuracy. 

An application of this method is illustrated in figure 

7 a. b. which as an example may describe a welding sta- 
tion in a production line for car bodies as shown in figure 

8 a.b, The under part of the car body is based on a fix- 
ture 24. while the side parts of the car body are held in 
place by fixtures 23. The mount of the car body parts to 
the fixtures are made by the use of steering pins and 
clamping mechanisms 27. These are controlling the 
location of each different component during the welding 
process, and have to be in the correct location to ensure 
correct shape of the assembled car body. Control of 
each individual object is efficiently made by the method 
described above (figure 6) by having a number of fixed 
help reference points distributed all over the whole sta- 
tion. An alternate solution is shown in figure 7 a. where 
the help reference points a - u are attached to a portable 
jig 24 that can be moved into the station when a meas- 
urement is to be made. The location of this jig is in such 
a case controlled by the main references 25. 26 of the 
station (e.g. steering pins) that are holding the jig in a 
fixed position. The positions and orientations of the two 
cameras 1 , 8 relative to the object f ixed coordinate sys- 
tem are first determined by registering the directions 
towards the known points a - u. The advantages of this 
method is that each individual station in the whole pro- 
duction line is based on a single jig with a fixed and well 
known geometry, such that a unique, accurate geome- 
try is ensured all along the production line. Hence, the 
error propagation through the production line is kept to 
a minimum. Furthermore, it will not be necessary to 
have an extensive network of points in each individual 
station. 

Figure 9 shows another application of the same 
principle. In this case the reference points are fixedly 
permanently mounted in a structure 28. while the object 



29 to be measured is transported into this structure. As 
an example this method can be applied in a production 
line based on transportable jigs for transport of compo- 
nents into the welding stations. Control of these jigs on 
5 a regular basis can be done in fixed measurement sta- 
tions. 

Figure 10 shows an example of this, where the 
object 30 is an aircraft that is brought to a measurement 
station where the reference points are placed on the 
10 f ioor or in a surrounding fixture. 

The system known per se shown in figure 3 must, to 
cover the methods described above, be extended with 
respect to the data processor 10. the camera control 
unit 1 1 and the light source control unit 12. In this cen- 
ts text it is required that these units shall be able to take 
into account: 

- different types and number of cameras. 
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30 



diff er ent types and number of light sources and 
help tools. 

different calculation methods depending on the 
configuration of cameras and light sources that are 
used and which type of known information (defined 
coordinate values in a local coordinate system or 
given distances between points) that should be 
included in the calculation. 



Some examples of applications are described 
above. These shall be considered as examples only, as 
a large number of different other objects may be meas- 
ured by the same methods and the same system. This 
includes large as well as small objects within automo- 
35 tive. aerospace or other mechanical industry. 

Claims 
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A method of determining the position and/or orien- 
tation of a number of objects relative to each other, 
characterized in that the method comprises the 
steps of: 

(A) providing at least two cameras; 

(B) providing a probing tool; 

(C) providing a network of help reference 
points comprising new measurement points in 
the form of active light sources, reflective points 
or points of measurement using the probing 
tool, and existing points or geometrical objects 
whose position can be related to a physical 
point; 

(D) determining the positions of elected ones of 
points in the network relative to each other by 
using at least one of the cameras positioned at 
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a plurality of arbitrarily elected places, the posi- 
tion ol light sources and light reflective points 
being registered in the form of their image on a 
sensor in said camera, and the imaged point 
being indicated in the form of coordinates 
related to a camera fixed coordinate system, 
the position of points or geometric objects the 
position of which can be related to a physical 
point being registered by holding said probing 
tool in contact with the point ad calculating the 
touch point image on the camera sensor based 
on the observed images of the probing tool light 
sources or reflective points, the position of the 
individual points relative to an arbitrary spatial 
coordinate system being calculated based on 
bundle adjustment. 

(E) determining the positions and orientations 
of elected ones of the camera by using the 
positions of the network points determined in 
step (D). 

(F) determining the position of elected ones of 
said objects relative to each other based on the 
positions being determined for said elected 
ones of the cameras, whereby the position of at 
least one of the objects is determined by 

(i) holding the probing tool in contact with 
at least one point on the object, and 

(ii) obtaining measurement data from the 
probing tool using at least two of the cam- 
eras whose positions were determined in 
step (E). 

2. The method as in claim 1 , characterized in that the 
probing tool has a touch point and wherein step 
(F)(i) comprises the step of holding the touch point 
of the probing tool in contact with the point on the 
object. 

3. The method as in claim 1, characterized in the 
steps of: 

positioning at least one reference bar within a 
measurement volume containing the objects; 
and 

obtaining scale information from the reference 
bar. 

4. The method of claim 1 . characterized in that at least 
some of the network points are light sources, and 
that the step (D) of determining the positions com- 
prises determining the positions of some of the net- 
work points light sources. 

5. The method claim 1 , characterized in that the posi- 
tion of at least one of the points in the network is 



determined using the probing tool. 

6. The method of claim 1, characterized in that the 
probing tool has light sources attached thereto. 

5 

7. The method of claim 4, characterized in that the 
light sources are chosen from active light sources 
and passive light sources. 

io 8. The method of claims 7, characterized in that the 
active light sources are chosen from fixedly 
mounted light sources, projected light spots, and 
illuminated reflectors. 

is 9. The method of claim 1, characterized in that the 
network includes at least two points having a known 
mutual separation distance, the two points being 
used to determine a correct length scale. 

20 10. The method of claim 1, characterized in the step of: 

determining optical properties of at least one of 
the cameras based on observed positions of 
the network points. 
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11. The method of claim 1. characterized in that the 
positions of some of the points in the network define 
a coordinate system. 

12. The method of claim 1 1 , characterized in that step 
(F) determines the position of each object relative 
to the coordinate system. 

13. The method of claim 1, characterized in that the 
network is established within a volume, and that 
objects are brought within the volume prior to the 
step of determining their positions. 

14. The method of claim 1, characterized in that the 
objects are used to locate and/or hold parts in a sta- 
tion in a production - or manufacturing line. 



15. The method of claim 14, characterized in that the 
network is established on a moveable object and 
45 that the movable object is moved into the station 
before performing step (F) . 



16. The method of claim 1, characterized in that the 
network is established on a moveable object, and 
that the movable object is moved into each of a plu- 
rality of stations in a production line and then per- 
forming step (F) in each of the stations. 
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17. A method according to claim 1. characterized in 
that the method operates relative to a defined coor- 
dinate system, that a minimum of three of the net- 
work points have at least one of their coordinate 
values known relative to the defined coordinate sys- 
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tern, such that a minimum of six coordinate values 
are Known and distributed, and such that three val- 
ues are defined relative to one coordinate axis of 
the defined coordinate system, two values are 
defined relative to a second coordinate axis of the 5 
defined coordinate system, and one value is 
defined relative to a third axis of the defined coordi- 
nate system, and that the positions and orientations 
of other points and geometrical objects relative to 
this defined coordinate system using the known to 
coordinate values. 

18. A method as in any one of the preceding claims, 
characterized in that said objects include: 

75 

geometrical objects. e.g. planes, lines, holes, 
cylinders, spheres, and/or combined objects 
including robot arms and industrial production 
equipment 
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Fig. 1a. (Prior art) 
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Fig. 1 b. (Prior art) 
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Fig. 2 (Prior art) 
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Fig. 4a 
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Fig. 6 
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□ FADED TEXT OR DRAWING 

□ BLURRED OR ILLEGIBLE TEXT OR DRAWING 

□ SKEWED/SLANTED IMAGES 

□ COLOR OR BLACK AND WHITE PHOTOGRAPHS 

□ GRAY SCALE DOCUMENTS 

□ LINES OR MARKS ON ORIGINAL DOCUMENT 

□ REFERENCE(S) OR EXHIBIT(S) SUBMITTED ARE POOR QUALITY 

□ OTHER: ^S^iolffi Hjl 

IMAGES ARE BEST AVAILABLE COPY. 
As rescanning these documents will not correct the image 
problems checked, please do not report these problems to 
the IFW Image Problem Mailbox. 
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